Milwaukee Longitudinal School Choice 
Evaluation: Annual School Testing 
Summary Report 2008-09 



.Jeffery R. Dean 

University of Arkansas 

Patrick J. Wolf 

University of Arkansas 



SCDP Milwaukee Evaluation 
Report #18 

April 2010 




The University of Arkansas 

was founded in 1871 as the flagship institution 
of higher education for the state of Arkansas. 
Established as a land grant university, its mandate was threefold: to teach students, conduct 
research, and perform service and outreach. 

The College of Education and Health Professions established the Department of Education 
Reform in 2005. The department’s mission is to advance education and economic development 
by focusing on the improvement of academic achievement in elementary and secondary schools. 
It conducts research and demonstration projects in five primary areas of reform: teacher quality, 
leadership, policy, accountability, and school choice. 

The School Choice Demonstration Project (SCDP), based within the Department of Education 
Reform, is an education research center devoted to the non-partisan study of the effects of school 
choice policy and is staffed by leading school choice researchers and scholars. Led by Dr. Patrick 
J. Wolf, Professor of Education Reform and Endowed 21st Century Chair in School Choice, 
SCDP’s national team of researchers, institutional research partners and staff are devoted to the 
rigorous evaluation of school choice programs and other school improvement efforts across the 
country. The SCDP is committed to raising and advancing the public’s understanding of the 
strengths and limitations of school choice policies and programs by conducting comprehensive 
research on what happens to students, families, schools and communities when more parents are 
allowed to choose their child’s school. 






Milwaukee Longitudinal School Choice Evaluation: 
Annual School Testing Summary Report (2008-09) 



Jeffery R. Dean 

University of Arkansas 

Patrick J. Wolf 

University of Arkansas 



SCDP Milwaukee Evaluation 
Report #1 8 

April 2010 

/s SCHOOL CHOICE 

DEMONSTRATION PROJECT 

School Choice Demonstration Project 
Department of Education Reform 
University of Arkansas 
201 Graduate Education Building 
Fayetteville, AR 72701 

479-575-6345 



http://www.uark.edu/ua/der/SCDP/Milwaukee_Research.html 



April 2010 



Executive Summary 

With the passage of the 2005 Wisconsin Act 125, private schools participating in the Milwaukee 
Parental Choice Program (MPCP) have been required to administer annual standardized tests in reading, 
mathematics, and science to their MPCP students enrolled in the 4th, 8th, and 10th grades. The law 
further directs Choice schools to submit copies of the scores from those tests to the School Choice 
Demonstration Project for processing and reporting to the Legislative Audit Bureau. During the 2008-09 
school year, MPCP schools administered either nationally normed tests, such as the Iowa Test of Basic 
Skills, or the state criterion-referenced Wisconsin Knowledge and Concepts Examinations (WKCE). 

The School Choice Demonstration Project (SCDP) received student test scores from 113 of the 116 
schools participating in the MPCP that were required to administer tests. Specifically, the SCDP received 
5,654 nationally normed student test scores and 1,134 WKCE test scores. Seventy-nine of 113 schools 
submitted only normed tests, 16 schools submitted only the WKCE, and 18 submitted both types of tests. 

Nationally normed test scores for MPCP students in grades 4, 8, and 10 averaged between the 30th and 
40th percentiles nationally, depending on grade and subject. These average percentile scores were slightly 
higher than those obtained on the National Assessment of Educational Progress (NAEP) for low-income 
students in large cities across the nation. In other words, MPCP students taking normed tests performed 
slightly better relative to the national population of students taking those tests, on average, than did a 
similar population of low-income urban students across the nation on the nationally normed NAEP test. 
Fourth grade students in the MPCP averaged around the 33rd percentile nationally in reading, math, and 
science. The MPCP students tested in 8th grade averaged around the 37th percentile in the same three 
subjects. The MPCP students tested in 10th grade averaged at the 38th percentile across reading, math, 
and science. 

Normed test scores for the cohorts of 4th, 8th, and 10th grade students tested in the 2008-09 school 
year were slightly higher overall than scores for the cohorts of students tested in the same grades in the 
previous school year. Within the MPCP, average scores across the three grades and three subjects are 
higher in 2008-09 for five of nine comparisons, and lower for four. Since test scores are drawn from the 
same grades in consecutive years, representing scores for different groups of students, these averages are 
not necessarily indicative of growth in individuals’ academic performance. 

MPCP students who took the WKCE can be compared approximately to similarly income-disadvantaged 
students in the Milwaukee Public Schools (MPS). Similar household income limits apply to participation 
in both the MPCP and the federal free- and reduced-price lunch program (FRL), thus allowing the low- 
income MPCP population of students to be compared to a group of low-income students in the public 
schools. The 4th grade MPCP students who took the WKCE on average scored 7 to 26 scaled score 
points (equal to .15 to .55 of a standard deviation) below the average scores of MPS FRL 4th graders. The 
8th grade MPCP students who took the WKCE performed better than MPS FRL students by 3 to 13 
scaled score points (.06 to .24 of a standard deviation). Only 96 Choice students in 10th grade took the 
WKCE, too few to generate reliable comparisons. 
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Readers are urged not to draw conclusions about the relative success of the MPCP or MPS based upon 
these rough descriptive comparisons. Any differences observed between the test scores of MPCP and 
MPS FRL students are open to varying interpretations. The higher scores observed for MPCP over 
MPS FRL students in the 8th grade could be attributed to older students in the MPCP benefiting 
from participation in the program, or it might be true that those 8th grade students who have remained 
in the MPCP are simply more able than those who have left. Moreover, the subset of MPCP schools 
that administer the WKCE as their accountability test may not be representative of the total population 
of MPCP schools. Any reliable determination of the effectiveness of a school choice program like the 
MPCP can only come from a rigorous experimental or longitudinal study that follows a representative 
group of choice students over time and compares their achievement gains to those of a comparable set of 
public school students. For such an evaluation we refer readers to the MPCP Longitudinal Educational 
Growth Study (LEGS) also being conducted by the SCDP 

This report and its companion reports are the third in a series of annual reports on the Milwaukee 
Parental Choice Program (MPCP) conducted by the School Choice Demonstration Project (SCDP). 

This ongoing research project is being funded by a diverse set of philanthropies including the Annie E. 
Casey, Joyce, Kern Family, Robertson, and Walton Family Foundations. We thank them for their generous 
support and acknowledge that the actual content of this report is solely the responsibility of the authors 
and does not necessarily reflect any official positions of the various funding organizations or the University 
of Arkansas. 
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Introduction 

On March 10, 2006, Wisconsin Governor Jim Doyle signed Wisconsin Act 125 into law. The Act 
modified several elements of the Milwaukee Parental Choice Program (MPCP), the nation’s first and 
largest urban school voucher program. Participating private schools are now required to administer either 
the WKCE (the Wisconsin state test) or a nationally normed standardized test annually in reading, 
mathematics, and science to their MPCP (a.k.a. “Choice”) students enrolled in the 4th, 8th, and 10th 
grades. Beginning in 2006 and extending through 2011, the individual student results of the tests must be 
provided to the School Choice Demonstration Project (SCDP). Finally, Act 125 requires that: 

The [Wisconsin] legislative audit bureau shall review and analyze the 
standardized test data received from the School Choice Demonstration Project. 

Based on its review, in 2007 and annually thereafter until 2011, the bureau shall 
report to the legislature under s. 13.172 (2) the result of the standardized 
tests administered under subd. 1., the scores of a representative sample of 
pupils participating in the program under ss. 118.30 and 121.02 (1) (r), and 
scores of a comparable group of pupils enrolled in the school district operating 
under this chapter on the tests under ss 118.30 abd 121.02 (1) (r). 1 

This report describes the results of the student testing conducted by MPCP schools during the 2008-09 
academic year. 2 The standardized test scores were collected from participating schools throughout the 
school year, with most of them arriving at the SCDP in the summer of 2009. The SCDP staff carefully 
entered these scores into a single database and delivered the data to the Legislative Audit Bureau (LAB) 
on December 30, 2009. 3 

Most of the test scores received from Choice schools were nationally normed tests such as the Terra 
Nova or the Iowa Test of Basic Skills (ITBS). These types of tests measure performance relative to other 
students by including questions meant to produce a full range of scores (i.e., very easy questions ranging 
to very difficult questions to separate the highest and lowest performing students), and performance is 
measured across a large national sample of students. By contrast, 34 private schools participating in the 
MPCP administered the Wisconsin criterion-referenced test, the Wisconsin Knowledge and Concepts 
Examinations (WKCE), either exclusively or in addition to nationally normed tests (Table 1). Even 



1 Wl Act 1 25, Sec. 8, 1 1 9.23 (7)(e), 2. 

2 Although this report is mandated by Act 1 25, the law specifies that it be conducted by an independent research 
organization (i.e. the SCDP) and financed by non-governmental sources. 

3 The majority of MPCP schools administered the standardized tests late in spring 2009, with some administering in the 
fall of 2008. The companies that produce the tests require several months to score them and send the test results to 
the schools. Since most schools operate with a minimum staff over the summer, in some cases the test results were not 
compiled and sent to the SCDP until late summer 2009. The test score database was entered and checked for errors 
between the date of receipt of test scores and the date on which test scores were received by the LAB. 
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though the producers of both norm- and criterion-referenced tests claim that the assessments cover the 
same subject domain, students likely exhibit different patterns of proficiency on the two types of tests 
due to differences in content, difficulty, and framework. The WKCE is only administered in Wisconsin; 
therefore, no national distribution exists to allow a direct performance comparison with the ITBS, Terra 
Nova, and other nationally normed tests. 

To account for these differences, distinct sections of this report present aggregated results from schools 
that administered nationally normed tests and separate results from schools that administered the 
WKCE. The report compares the performance of Choice students and schools administering the WKCE 
with student- and school-level WKCE test scores for Milwaukee Public School (MPS) students who 
participate in the federal free- and reduced-lunch (FRL) program. 4 



Table 1 : Types of Tests Taken by MPCP Schools and Students 





Schools 


Students 


Number 


Percentage 


Number 


Percentage 


Nationally Normed Only 


79 


69.9% 


5,654 


83.0% 


WKCE Only 


16 


14.2% 


720 


10.6% 


Both Types Given* 


18 


15.9% 


434 


6.4% 


Total 


113 


100.0% 


6,808 


100.0% 



* For schools, this category broadly includes schools which gave both types of tests, though not necessarily to the same students. 
For example, some grades may have been given nationally normed tests, and others the WKCE, in which case each student in the 
school may have only taken one test. For students, however, this category includes only individual students who took both types of 
tests in the 2008-09 school year. 



The scores received from the MPCP schools are summarized in two results sections below. The first 
section describes the aggregate student-level scores for the groups of Choice students in each of the tested 
grades who took either nationally normed assessments or the WKCE. The second section presents the 
distribution of MPCP test scores, by grade and subject matter, averaged at the school level. 



4 As a mechanism for comparing MPCP and MPS students, eligibility for the federal lunch program is limited in two ways. 
First, the family income ceiling for eligibility for the lunch program is 185 percent of the poverty line, which is slightly 
higher than the income ceiling of 175 percent of poverty for initial eligibility for the MPCP but somewhat below the 
income ceiling of 220 percent of poverty for renewal of MPCP eligibility. Second, many students who are income- 
eligible for the federal lunch program choose not to participate. The rate of non-participation tends to increase steadily 
as students move from the lower grades to the higher grades. Although federal lunch program participation is an 
imperfect measure of family disadvantage, it was the best criterion available to generate approximate comparisons 
for this particular element of the evaluation. For this and other reasons described in this report, readers are cautioned 
against drawing any strong conclusions about the relative performance of MPCP and MPS students from the descriptive 
comparisons provided here. 
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Although school-level test scores are presented in the second results section, individual schools are not 
identified by name. Connecting any individual Choice school explicitly to information about its students 
could enable readers to identify individual participants in the study in violation of the assurances of 
confidentiality that are required when conducting such research. 5 The preservation of the anonymity of 
participants in educational evaluations is so important that the federal statute establishing the evaluation 
division of the U.S. Department of Education expressly prohibits the naming of individual students, 
parents, or schools in any of its reports. 6 Because state law requires the affected MPCP schools to 
administer tests and submit scores to the SCDP, we do mention by name in this report the schools that 
did and did not perform those required actions (Appendix A). Because the information submitted by 
the schools is designed to facilitate an education evaluation, however, any subsequent presentation of the 
data provided by schools and students must remain anonymous. Of the hundreds of statistical studies 
of school choice programs with which the authors are familiar, none of them have revealed school-level 
information directly associated with named schools for these very reasons. 7 

The MPCP Annual School Testing Summary Report has important strengths and limitations. The main 
strength of the Report is its ability to provide a data-rich snapshot of the current academic performance 
of a large number of students from nearly all of the schools participating in the MPCP. Such information 
on the Choice program has not been available for more than a decade. There are two primary limitations 
to this report. First, students are not required to test in consecutive grades, so currently there is no way 
to observe year-over-year changes in individual test scores. Differences in test score averages between 
the previous year’s report and the current report reflect the achievement of different cohorts of students. 
Students who were in grades 4, 8, and 10 in 2007-08 and have advanced are not included in the current 
2008-09 report. Since testing is not required for consecutive grades, no individual gains are observable 
in consecutive annual reports. Second, the comparison of MPCP and MPS FRL students is not ideal. 
Though participation in both programs is subject to income limits, the degree to which these limits reflect 



5 As with all academic research with human participants, the SCDP research team had to gain approval of an extensive 
protocol for protecting the anonymity of participants and the confidentiality of the information that they provide before 
research on the MPCP could begin. Approvals of our research protocols, which prohibit us from associating any data 
with named individuals or schools, were obtained from the Institutional Review Boards for Human Subjects Research at 
the University of Arkansas, the University of Wisconsin, the University of Kentucky, and Westat. 

6 "The Director shall ensure that all individually identifiable information about students, their academic achievements, 
their families, and information with respect to individual schools, shall remain confidential... "See Education Sciences 
Reform Act of 2002, 20 U.S.C., 1 232g, 1 232h. 

7 Many schools in the MPCP provide school-level information by school name voluntarily to parents and organizations to 
facilitate the school choice process. In Milwaukee and in other major cities around the country, the organization Great 
Schools Dot Net is spearheading efforts to consolidate such information into comprehensive school choice guides 
called "My School Chooser." Because such efforts are voluntary and are not part of a research evaluation, the prohibition 
against connecting descriptive information to named schools does not apply to such school choice guides. Copies of 
the "Milwaukee School Chooser"are available at: http://www.greatschools.net/geo/landing/milwaukee.page. 
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the actual demographics of the two groups is not knowable using the data received from these schools. 
Further, the performance of the two groups cannot be considered a reflection of the effectiveness of either 
the MPCP or the MPS, as the free selection of students into the two groups precludes a strict scientific 
comparison. Altogether, these data show us how well a large group of MPCP students is performing 
academically, but tell us nothing about what has caused them to perform at that level. Since many 
factors including the backgrounds and home lives of students as well as the quality of their educational 
experiences likely influence their performance on standardized tests at a given point in time, it would 
be a mistake for readers to draw conclusions concerning the effectiveness of the MPCP based on these 
simple annual descriptive statistics. At present, though this report offers a thorough description of MPCP 
student achievement, it can show neither progress in the learning of individual students, nor whether the 
MPCP as a program is more or less effective at educating students than the MPS. 

The Longitudinal Educational Growth Study (LEGS), also being conducted by the SCDP, overcomes 
some of the limitations of this report. By tracking student achievement longitudinally, rather than giving 
a series of cross-sectional snapshots of achievement, the report is better able to assess the effect over time 
of the MPCP on individual student achievement. It contains a comparison of the achievement gains 
over time of a representative sample of MPCP students relative to a carefully-matched set of peers in 
Milwaukee traditional public and charter schools. It tracks the performance of the same set of MPCP 
and MPS students as they progress through their education. This report, by contrast, examines a different 
set of MPCP students each year at fixed points in their educational experience. It is not a growth or 
value added comparison against peers in MPS or any other group. Readers who are interested in student 
performance differences that can be reasonably attributed to the influence of the Choice program itself are 
advised to follow the progress of the MPCP Longitudinal Educational Growth Study. 

Process for Obtaining MPCP Test Scores 

The School Choice Demonstration Project has performed a variety of responsibilities over the past three 
years to make this report possible. For the two previous annual testing summary reports, schools were 
notified, well in advance of testing deadlines, of the requirement that they submit test scores to the SCDP. 
After successive reminders, in both previous years the SCDP has achieved a response rate of 98-99 
percent of affected MPCP schools submitting acceptable test scores. 

On February 3, 2009, representatives of the SCDP attended the Pupil Assignment Council meeting for 
the 2008-09 school year. At this meeting, attending school leaders were reminded of their requirement 
to submit 2008-09 test scores in order to continue participation in the MPCP and mailing materials were 
distributed to facilitate the sending of schools’ test scores to the SCDP. School leaders were given a test 
score submission deadline of July 1, 2009. A reminder of this deadline was sent to schools on June 22. By 
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August, 113 of 116 affected schools had sent test scores for the 2008-09 school year. 8 Similar to previous 
years, this represents a response rate of over 97 percent. 

For the purpose of this report, the SCDP has had to distinguish in its database students who participated 
in the MPCP from non-voucher students attending MPCP-participating schools. To identify Choice 
students previously, the SCDP used an enrollment list, containing names, birthdates, and other personal 
information obtained from the Wisconsin Department of Public Instruction (DPI). This enrollment list 
has been matched to student information in the database to distinguish MPCP from non-participating 
student test scores, after which only MPCP students’ test scores have been used for the report. For legal 
reasons, the DPI enrollment list was not available to the SCDP for the 2008-09 school year. 9 Lacking 
access to the 2008-09 DPI Choice enrollment list, we used the previous year’s DPI list to identify the 
Choice students in the test score database. The current report thus identifies all tested students in 2008- 
09 who participated in the MPCP in 2007-08. The only tested Choice students not identified for 2008- 
09 were those who were first-time voucher recipients in the 2008-09 school year. 

Use of Percentile Rankings for Nationally Normed Tests 

Act 125 requires that schools participating in the MPCP administer either the WKCE or a nationally 
normed standardized test. Schools which chose to administer nationally normed tests used a variety of 
tests. For the descriptive purposes of this report, these schools’ various normed test scores are combined 
and presented using a single metric which suffices to similarly describe student achievement across 
different tests. 

Though scaled scores are often the preferred metric for reporting test scores, they cannot be easily 
combined across tests to produce an aggregate snapshot of student performance. A scaled score of 300 
on one test might fall at the lower end of the performance distribution, while on another test it might be 
a very high score. For common norm-referenced tests such as the ITBS and the Terra Nova, the scaled 
scores for corresponding national percentiles are drastically different. For example, on the 4th grade test, 
the corresponding scaled score for the 50th percentile on the ITBS is 200. For the 4th grade Terra Nova, 
the 50th percentile scaled score is 637. Due to this variation, scaled scores are inappropriate for averaging 
across nationally normed tests. Instead, for all such tests, National Percentile Ranks (NPRs) are used in 



8 A school was "affected" if it enrolled at least one MPCP student in grades 4, 8, or 10. A total of 1 14 affected schools 
submitted student test scores, but one school only provided scores that were unacceptable because they were neither 
from a norm referenced test nor from the WKCE. 

9 On August 14 th , 2009, Dr. Wolf received notice from the Department of Public Instruction that the full enrollment list 
for schools participating in the MPCP would, for legal reasons, be unavailable for the 2008-09 school year. The DPI is 
responsible for maintaining student and family confidentiality with their data, and it saw no warrant in existing statute 
for the release of these personal data to the SCDP for the current report. 
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this report. All participating schools which submitted nationally normed tests reported student NPRs in 
reading, math, and science. These scores produced the aggregate performance totals given below. 10 

In the case of the WKCE, no national norm or percentile exists. Due to the Wisconsin-specific nature 
of this test, this report presents student test scores in terms of scaled scores for MPCP schools which 
administered the WKCE. Because MPS also administers the WKCE, this report also uses test scores for 
MPS FRL students as a comparison to the test scores of similarly low-income MPCP students. 

Percentile Results at the Program/Grade/Subject Level 

Here we report national percentile rank (NPR) averages for MPCP students in grades 4, 8, and 10 in the 
subjects of reading, math, and science for the two most recent years of data which the SCDP has received. 
Readers are cautioned about comparing one year’s results with another as different students took these 
tests in different years. These results are “repeated cross-sectional” and not longitudinal in nature. 

In 2009, the SCDP received scores for 940 4th grade MPCP students. 11 Of these, 670 took a nationally 
normed test. These students scored on average between the 30th and 35th percentile nationally in all 
three subject areas (Table 2 and Figure 1). These particular scores do not indicate how well the program 
serves MPCP students; they merely describe the average performance of an educationally disadvantaged 
group of students at a particular moment in time relative to the average student in the nation, most of 
whom are not as disadvantaged. To interpret the results, one would say that the 4th grade Milwaukee 
Choice students taking a nationally standardized test averaged near the 35th percentile (scored higher 
than 35 percent) in reading compared to other 4th graders nationwide taking similar tests. Choice 
students in the 4th grade averaged in the 30th national percentile in math and the 34th national percentile 
in science. 



10 Although all of these scores are similar in that they describe the student's performance in comparison with the national 
sample of students that took the test (i.e., the "norming" population), that national sample can vary somewhat across 
the tests - another reason why readers are cautioned against drawing strong conclusions from these illustrative data. 

1 1 The total number of student test scores by grade reported here and throughout the text are counts of the number of 
different students for whom at least one test score on a norm-referenced test or the WKCE was provided. That number 
for each grade is higher than the "Observations" numbers reported in each table because we separate out the WKCE 
scaled scores from the norm-referenced percentile scores and because most but not all of the students produced test 
scores in all three of the subject areas. 
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Table 2: 2007-08 and 2008-09 MPCP National Percentile Summary Statistics 





2007-08 


2008-09 


Reading 


Math 


Science 


Reading 


Math 


Science 


4th Grade 


Mean 


37.8 


35.0 


32.9 


34.9 


29.7 


33.8 


Obs 


654 


648 


701 


669 


664 


663 


Std. Dev. 


25.5 


24.8 


24.3 


24.8 


25.0 


23.1 


8th Grade 


Mean 


36.3 


36.1 


33.8 


35.7 


37.2 


36.7 


Obs 


443 


436 


485 


533 


530 


528 


Std. Dev. 


25.2 


25.8 


22.3 


22.4 


26.1 


22.6 


10th Grade 


Mean 


40.7 


35.1 


35.2 


39.4 


38.4 


36.7 


Obs 


508 


531 


518 


464 


467 


464 


Std. Dev. 


25.8 


22.4 


25.4 


25.0 


23.9 


23.8 



Fig. 1 : Avg. NPRs for MPCP Students Taking Nationally Normed Tests, 4th Grade 
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The MPCP students tested in 8th grade performed similarly (Table 2 and Figure 2). The SCDP received 
scores for 856 8th graders in the MPCP for the 2008-09 school year. Of these, 533 took a nationally 
normed test. Choice students in 8th grade, on average, performed slightly above the 35th percentile 
nationally in reading, math, and science. 
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Test scores were received for 565 MPCP students in the 10th grade. Of these, 467 took a nationally 
normed test. This group scored the highest relative to national norms of the three grade-cohorts tested 
(Figure 3 and Table 2). The 10th grade Choice students scored near the 40th percentile in reading, 
the 38th in math, and the 37th in science. Although these descriptive statistics appear to show some 
academic improvement as Choice students mature, these grade-cohorts of students are compositionally 
different. Moreover, these data tell nothing about individuals’ achievement growth over time, since they 
measure different groups of students at the same point in time. Readers are cautioned against inferring 
from these data that the MPCP is responsible for the difference in performance between 10th grade 
MPCP students and their 8th and 4th grade counterparts. 

Fig. 3: Avg. NPRs for MPCP Students Taking Nationally Normed Tests, 10th Grade 
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In total, MPCP students taking normed tests in grades 4, 8, and 10 averaged between the 30th and 40th 
percentiles nationally across grades and subjects. Given the low socioeconomic status (SES) of these 
students, these below-average test scores are not unusual. However, an estimation of their performance 
relative to similarly-disadvantaged students could illuminate whether these students are performing 
at a level expected given their low SES as well as their location in a large urban environment. While 
imperfect, such an estimation can be obtained by comparing these students’ NPRs to the average NPRs 
for a similar national group of students on the well-known National Assessment of Educational Progress 
(NAEP). Fortunately, such data are available, allowing a comparison of NPR averages for MPCP 
students to the most recent NAEP scores for FRL students in large cities nationwide. 

Figure 4. MPCP Normed Tests and Large City FRL NAEP, Grade 4 
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Figure 5. MPCP Normed Tests and Large City FRL NAEP, Grade 8 
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As can be seen from Figures 4 and 5, MPCP students in tested grades who took normed tests scored as 
well or better than a similar national population of students who most recently took the NAEP Though 
comparisons are made for grades 4 and 8, such a comparison for MPCP students in grade 10 could not be 
made, as NAEP did not test in that grade. 

While these results provide a rough estimate of how MPCP students are performing relative to similar 
students nationally, readers are cautioned that the comparison is imperfect. The test scores are drawn from 
different years, the most recent NAEP administration in 2007 and the most recent MPCP administration 
in 2008-09. The scores are drawn from different tests, though all tests are normed from very large national 
samples. Finally, how similar the two groups are demographically cannot be precisely determined at 
present, though both have met a similar income threshold to qualify for a government program. These 
facts should be kept in mind when interpreting the numbers in Figures 4 and 5. 

Scaled score Results at the Program/Grade/Subject Level 

The fact that a subset of MPCP students as well as MPS students took the WKCE criterion-referenced 
test allows us to present those results in a different manner. MPS FRL students are likely to be more 
similar to MPCP students than any national 
norming population, since both groups of 
students live in the same city and qualify as 
low income. Still, because this approximate 
match is not very precise, and because the 
subset of MPCP students who took the 
WKCE is not necessarily representative of 
all MPCP students, readers are cautioned 
against drawing conclusions about the 
effects of the Choice program from this 
snapshot comparison. 

Table 3 illustrates the summary statistics 
for 4th and 8th grade MPCP students who 
took the WKCE in the fall of 2008 as well 
as the statistics from similar MPS FRL 
students. Figures 4 and 5 show scaled 
score differences between the comparison 
groups, including tests for whether these 
differences are statistically significant. 
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Table 3: 2007-08 and 2008-09 WKCE Summary Statistics for Scaled Scores: Grades 4 and 8 

Reading Math Science 

MPCP MPS MPCP MPS MPCP MPS 





07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


4th Grade 


























Mean 


429 


435 


438 


442 


411 


412 


429 


438 


272 


259 


272 


272 


Obs 


457 


262 


4723 


4548 


465 


262 


4825 


4611 


409 


262 


4810 


4586 


Std. Dev. 


56 


43 


51 


47 


53 


51 


50 


47 


61 


31 


29 


30 


8th Grade 


























Mean 


486 


496 


480 


483 


489 


503 


488 


500 


368 


372 


361 


361 


Obs 


579 


318 


4641 


4151 


581 


318 


4657 


4234 


561 


317 


4595 


4186 


Std. Dev. 


52 


52 


55 


54 


53 


50 


54 


51 


51 


36 


38 


39 



Fig. 6: Grade 4 WKCE Scaled Scores for MPCP and MPS FRL Students 




□ MPCP 
■ MPS FRL 



These results show 4th graders in the MPCP who took the WKCE performing 7 to 26 scale points 
below the average levels of MPS FRL 4th graders. This achievement difference is equal to .15 to .55 of a 
standard deviation of the MPS test score distribution. As shown by the p-values at the bottom of Figure 
6, these differences are great enough to be considered statistically significant, such that the differences 
observed are highly unlikely to be due to chance. These differences, as cautioned before, should not be 
considered a reflection of the relative effectiveness of either the MPCP or the MPS. 
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The 8th grade scores in Figure 7 show a pattern opposite to that of the 4th grade scores. Scaled score 
averages for MPCP students in 8th grade on the WKCE are higher in all three subjects, by 3 to 13 scaled 
score points, than those of MPS FRL 8th grade students. This achievement difference at the 8th grade 
level is equal to .06 to .24 of a standard deviation. Of the three differences observed, those in reading and 
math are shown to be statistically significant at a very high level (p=.00), whereas the difference between 
the two groups in math is too small to preclude the possibility that it is the result of random variation. 

The differences in test score patterns for the 4th and 8th grade comparisons is similar to that observed in 
previous reports. In the 2007-08 testing report, MPCP 4th graders taking the WKCE scored lower on 
average than MPS FRL students in all three subjects. In the 8th grade, MPCP students scored higher 
than MPS FRL students. These differences are similar across years despite the fact that the current report 
examines a different cohort of students than the previous report in both 4th and 8th grades. Causes of this 
consistent variation are uncertain. Despite the uncertainty, the persistence of similar patterns across years 
suggests that the variation may not be random. 

Only 96 MPCP 10th graders took the WKCE in 2008-09, making aggregate statistics about that small 
subgroup insufficiently reliable to present here. 

Performance Distributions at the Individual School Level 

This section presents a series of histograms that describe the distribution of results of the 2008-09 MPCP 
school testing at the school level. Rather than aggregating all MPCP students as a single population 
and providing averages for that population, this section treats MPCP school-level averages as the unit 
of analysis. The histograms illustrate 4th and 8th grade scores in reading, math, and science. Figure 8 
represents 4th grade and Figure 9 shows 8th grade in all three subjects. These histograms have a normal 
distribution overlay, which is depicted by the line in each graph. This normal distribution overlay is 
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relative to the empirical data that underlay it. That means it is appropriately scaled and has the same 
mean and standard deviation as the data. These histograms allow for slightly more disaggregation. For 
example, as mentioned earlier, the 4th grade reading percentile mean was 36.8, indicating students on 
average ranked just above the lower one-third of the distribution. Looking at school averages in the 4th 
grade reading histogram, however, there were 12 MPCP schools whose Choice students averaged higher 
than the 50th percentile in their test score performance. 

These particular histograms use frequency counts of the number of schools with average percentile scores 
in each decile, which is a range of ten percentiles. For example, in the histogram for 4th grade reading, the 
Choice students in 14 schools had an average national percentile score in Grade 4 reading between the 
40th and 50th percentiles. In general, these graphs are intended to indicate clusters of low-performing, 
average-performing, and high-performing MPCP students grouped at the school level. 

Figure 8. 4 th Grade National Percentile Rank for MPCP Schools 

FIGURE 8a FIGURE 8b 

MPCP SCHOOL AVERAGE NPR, GRADE 4 READING MPCP SCHOOL AVERAGE NPR, GRADE 4 MATH 




FIGURE 8c 

MPCP SCHOOL AVERAGE NPR, GRADE 4 SCIENCE 
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Figure 9. 8 th Grade National Percentile Rank for MPCP Schools 

FIGURE 9a FIGURE 9b 

MPCP SCHOOL AVERAGE NPR, GRADE 8 READING MPCP SCHOOL AVERAGE NPR, GRADE 8 MATH 





FIGURE 9c 

MPCP SCHOOL AVERAGE NPR, GRADE 8 SCIENCE 




The histograms of MPCP school-level test scores presented above generally show distributions with 
what is known as a “positive skew.” The school-level performance averages tend to cluster around the 
30th and 40th percentiles, with a small group of much higher performing units at the upper tail of the 
distribution. The positive skew is visibly present in both grades 4 and 8, indicating that a small number 
of schools raise the school-level averages on these scores. The distributions reveal that some school-level 
clusters of MPCP students are performing very well relative to national norms; however, most school- 
level clusters of students performed just below national averages. Because these data merely provide a 
snapshot of school-level groupings of students, we cannot infer from them that some MPCP schools are 
much better performing than most MPCP schools, since the results could be due to higher-performing 
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groups ofMPCP students gravitating towards particular MPCP schools. In other words, we cannot rule 
out student self-sorting as the cause of the school-level performance distributions presented above. 

Table 4 illustrates the WKCE scaled score data aggregated to the school level. Although similar to Table 
3, one can see how statistics change when test scores are distributed among schools and then averaged 
at the school level as opposed to aggregated across an entire program or school system. Under almost 
all circumstances, the averages of subgroup averages (e.g., performance by school) will provide different 
results than taking the total average of the population (e.g., all testers). 12 Because a small number of 
MPCP schools are performing well above both the MPCP and MPS system-level averages, the WKCE 
scores averaged at the school level result in comparisons somewhat more favorable to the MPCP schools 
than the comparisons based on test-scores at the individual student level. Still, the performance of 
school-level groupings of 4th grade Choice students remains lower than the performance of school-level 
groupings of 4th grade MPS FRL students. While test score averages were identical for MPS FRL and 
MPCP schools in 4th grade reading, MPCP students underperformed MPS FRL students by .11 and .42 
standard deviations respectively for math and science. A similar comparison of school-level results in 8th 
grade shows that MPCP student performance averaged at the school level is higher than that of 8th grade 
MPS FRL students by 16 to 28 scaled score points (.37 to .60 of a standard deviation). 



Table 4: 2007-08 and 2008-09 WKCE Summary Statistics for Scaled Scores, School Level 
Reading Math Science 





MPCP 


MPS FRL 


MPCP 


MPS FRL 


MPCP 


MPS FRL 




07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


07-08 


08-09 


4th Grade 


























Mean 


433 


444 


439 


444 


416 


421 


430 


439 


271 


270 


273 


273 


Obs 


23 


20 


118 


118 


23 


20 


118 


118 


23 


20 


118 


118 


Std. Dev. 


34 


46 


18 


43 


40 


44 


19 


43 


41 


29 


12 


28 


8th Grade 


























Mean 


494 


506 


478 


478 


498 


511 


485 


495 


365 


370 


358 


358 


Obs 


27 


25 


90 


90 


27 


26 


90 


90 


27 


25 


90 


90 


Std. Dev. 


34 


43 


28 


47 


38 


32 


29 


43 


23 


28 


21 


34 



The following 12 histograms graphically show the Table 4 data for those MPCP schools taking the 
WKCE as well as school-level averages for FRL students in the MPS. 



12 In statistical parlance, the fact that an average of subgroup averages rarely equals the average of the entire group is 
called the "ecological inference" phenomenon. 
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Figure 1 0. 4th Grade WKCE Reading Histograms for MPCP and MPS Schools 



FIGURE 10a FIGURE 10b 

MPCP SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 READING MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 READING 





As given above in Figure 10, the means of the two populations are almost identical, but the MPCP group 
has a slightly higher standard deviation (e.g., greater variation in performance) and has more schools in 
both the upper and lower tail of the distribution. We would expect this wider variation in school-level 
achievement among the variegated set of MPCP schools. 



Figure 1 1 . 4th Grade WKCE Math Histograms for MPCP and MPS Schools 



FIGURE 10a 

MPCP SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 MATH 



FIGURE 10b 

MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 MATH 





Figure 1 1 shows that the pattern for 4th grade math is similar to that above for reading. The mean of 
MPCP school-level scaled scores is lower than for MPS FRL, while the standard deviation of MPS FRL 
scaled score averages is lower than that for the MPCP. 
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Figure 12. 4th Grade WKCE Science Histograms for MPCP and MPS Schools 

FIGURE 12a FIGURE 12b 

MPCP SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 SCIENCE MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 4 SCIENCE 





The science scores (Figure 12) have the highest standard deviation difference between the two groups, 
while the mean difference is very small. The large difference in standard deviation is likely due to the 
effect of two high-performing MPCP schools on the standard deviation of the MPCP sample. With the 
small number of schools in the MPCP program, these two schools of higher-performing students skew 
the mean and standard deviation results higher at the school level. These types of results are expected in 
such an analysis. 



Figure 1 3. 8th Grade WKCE Reading Histograms for MPCP and MPS Schools 



FIGURE 13a FIGURE 13b 

SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 READING MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 MATH 





In the case of 8th grade reading, shown in Figure 13, a greater proportion of FRL students in MPS 
schools averaged near the mean, resulting in a lower standard deviation for MPS schools than for MPCP 
schools. The MPCP histogram shows a higher mean than the MPS figure, by 28 scaled score points. 



Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2008-09 





Figure 14. 8th Grade WKCE Math and Science Histograms for MPCP and MPS Schools 



20 



April 2010 



FIGURE 14a 

MPCP SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 MATH 




FIGURE 14b 

MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 MATH 




FIGURE 14c 

MPCP SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 SCIENCE 




FIGURE 14d 

MPS FRL SCHOOL AVERAGE WKCE SCALED SCORES, GRADE 8 SCIENCE 




The math and science histograms for 8th grade (Figure 14) also show relatively normal distributions for 
each group. For both MPCP and MPS FRL school-level distributions, the mean school-level average, an 
average of averages, is slightly higher for MPCP than for MPS. 

These tables and histograms suggest there are schools in each group (MPCP and MPS FRL) in which 
students are performing well above or well below the MPS FRL group average. These results suggest 
there is ample variation in both groups. The standard deviations for the MPCP group are often higher, 
which is likely because of the lower number of schools and greater diversity of schools in that group. 

One should be careful in interpreting these data. The differences across the comparison groups and 
between the grades cannot necessarily be attributed to the Choice program. The figures are presented 
merely to provide a general description of the school-level performance of MPCP students and place that 
performance in context by making a rough comparison with the school-level performance of income- 
disadvantaged MPS students. 
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Proficiency Score Results 

Scaled scores, however, are not the only way to report results from criterion referenced tests such as the 
WKCE. States and schools often report scores in terms of proficiency rates. That is, if a student received 
a specific scaled score or higher, then the student is proficient in that particular subject area. This section 
of the report shows the MPCP students’ results in terms of proficiency percentages. Table 5 provides the 
percent of students proficient in each grade for each subject for MPCP and the FRL students of MPS. 

Table 5 shows that on the 4th grade reading test, 58 percent of MPCP 4th graders for whom the SCDP 
received WKCE scores are proficient while 56 percent of MPS free and reduced lunch students are 
proficient. In 4th grade math, MPS FRL students exhibit a higher proficiency rate than their MPCP 
counterparts, and the two groups show identical proficiency rates in science. However, the story is again 
different for 8th graders. In all three subjects, MPCP students show markedly higher rates of proficiency 
than do MPS FRL students, with the difference in rates varying from a 14 percentage point advantage 
in science to nine percentage points in math. Viewing WKCE scores in terms of proficiency percentages 
thus indicates a similar story as described by Table 3, with one caveat. In 4th grade, MPS students only 
demonstrate higher WKCE proficiency rates than MPCP students in math. The MPCP 4th graders 
have higher proficiency rates than similarly income disadvantaged MPS students in reading, and the 
two groups have identical proficiency rates in science. However, the 8th grade results regarding WKCE 
proficiency are similar to the 8th grade results regarding NPRs. MPCP students score higher than their 
counterparts in MPS in all three tested subject areas. Figures 15 and 16 show a graphical representation 
of the differences between 4th and 8th grade reading and math respectively. 




Milwaukee Longitudinal School Choice Evaluation: Annual School Testing Summary Report 2008-09 







Percent Proficient Percent Proficient 



1 00 % 



22 



April 2010 



Figure 15. 

MPCP 8th Grade Reading Percent Proficient by School 




— 8th Grade Reading 
Percent Proficient 



— MPS FRL Average 



Figure 16. 

MPCP 8th Grade Math Percent Proficient by School 




— 8th Grade Math — MPS FRL Average 

Percent Proficient 
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As before, it must be said that the differences shown here are for two different groups of students; one 
cannot infer relative improvement in MPCP student proficiency rates over time, nor can one draw 
conclusions from these data regarding the effectiveness of the MPCP relative to the MPS. 

Yet another way to look at the percent proficient data is to aggregate it to the school level. The following 
four figures show the percent proficient in each MPCP school with an indication of the MPS school 
average for FRL students in that particular grade and subject. 

As indicated by the red line in Figure 17, 56 percent of MPS free and reduced lunch students are 
proficient in 4th grade reading. Of the 20 MPCP schools for which the SCDP received WKCE 4th 
grade reading scores, 11 had percentages of proficiency above 55 percent. Figure 18 shows 8th grade 
reading scores in the same manner. 



Figure 17 



MPCP 8th Grade Science Percent Proficient by School 




— 8th Grade Science — MPS FRL Average 
Percent Proficient 
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MPCP 4th Grade Reading Percent Proficient by School 




9 4th Grade Reading — MPS FRL Average 
Percent Proficient 



Of the 26 MPCP schools reporting WKCE 8th grade reading scores, 16 scored higher than the 59% 
average for free- and reduced-lunch students in MPS. 

The figures below showing the math comparison are most telling. Figure 19 indicates that only six of 
the 20 MPCP schools reporting such scores have a proficiency rate above that of MPS FRL students as 
a whole. Figure 20 paints a very different picture for 8th grade math, with 13 out of 26 MPCP schools 
having proficiency rates above that of FRL students in MPS. 
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Figure 19 

MPCP 4th Grade Math Percent Proficient by School 
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Figure 20 

MPCP 4th Grade Science Percent Proficient by School 
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The usefulness of these particular figures is to indicate the wide variation in test scores for students 
attending MPCP schools. Similar to the WKCE histograms above, MPCP schools serve a range of 
students with diverse abilities, and these charts serve to illustrate this variation. 



Summary and Recommendations 

The purpose of this report is to provide descriptive data regarding the test scores of Milwaukee Parental 
Choice Program students in grades 4, 8 and 10 in reading, math, and science, as reported to the 
School Choice Demonstration Project 2008-2009. The above tables, graphs, and histograms provide a 
snapshot of these students’ percentile scores relative to overall national norms, and scaled scores on the 
WKCE relative to MPS FRL students. Because national norms are based on students with “average” 
educational circumstances, and the MPCP exclusively serves low-income inner-city students, the fact 
that their average levels of performance on norm referenced tests cluster around the lower one-third of 
the distribution is not surprising and should not be interpreted to indicate that the MPCP in general 
or MPCP schools in particular are necessarily doing a poor job of educating students. As shown by the 
comparison with the NAEP percentile ranks, students in the MPCP are performing at higher levels than 
similar low-income students in large cities in the U.S. 

The comparison of the scaled scores of the Choice students and schools that took the WKCE with the 
scores of MPS FRE students and schools similarly is presented descriptively with no claim that the 
schools themselves were independently responsible for the various results. Any reliable determination of 
the effectiveness of a school choice program like the MPCP can only come from a rigorous longitudinal 
study that follows a representative group of choice students over time and compares their achievement 
gains to those of a comparable set of public school students. The MPCP Longitudinal Education Growth 
Study will serve as the proper source for such a determination. 
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The Longitudinal Evaluation of the Milwaukee Parental Choice Program 

Frequently Asked Questions 

Why do we have to test MPCP students? 

The testing requirements in the Milwaukee Parental Choice Program (MPCP) are the result of 2005 Act 
125 passed by the state legislature and signed into law on March 10, 2006. The new law also raises the cap 
on total participation in the MPCP to 22,500 students and requires that all schools participating in the 
MPCP obtain accreditation, among other changes. 

Act 125 outlines the two components of the required testing program: program accountability and 
program evaluation . MPCP schools have vital responsibilities in both of these areas. 

COMPONENT ONE: PROGRAM ACCOUNTABILITY 

1.1 Who and what are we required to test for program accountability purposes? 

Under the new law, all MPCP students in grades 4, 8, and 10 must be tested annually. The test must cover 
reading, math, and science. 

1.2 What test can we use? 

Schools can use any nationally normed standardized test. Qualified tests would include the Iowa Test of 
Basic Skills, SAT 10,TerraNova, or the Wisconsin Knowledge and Concepts Exam (WKCE). 

1.3 Who pays for the test? 

The participating MPCP schools are expected to pay for the costs of purchasing and scoring the test that 
they choose to administer. 

1.4 When should we test? 

Schools that choose to administer the WKCE for accountability purposes need to test during the fall 
testing window of October 27 th through November 28 th . Schools that choose to administer a national 
norm-referenced test can do so in the fall or spring. 

1.5 What testing information do we need to provide to whom for program accountability? 
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Act 125 requires that, beginning in 2006, all MPCP schools forward to the School Choice Demonstration 
Project (SCDP) the individual level results of all standardized tests that it administers. That means that 
if the school tests all students in grades 4, 8, and 10 - not just MPCP students - the scores for all those 
students must be provided to the SCDP Also, if a participating MPCP school administers standardized 
tests to students in grades besides 4, 8, and 10, the results of those tests must also be forwarded to the 
SCDP The basic requirements of Act 125 regarding testing are that participating schools must test all 4 th , 
8 th , and 10 th grade MPCP students annually, can test non-MPCP students and MPCP students in other 
grades, and must forward to the SCDP the individual-level results of any standardized testing that they 
conduct. 

1.6 Why is testing information being provided to the SCDP? 

Through Act 125, the state government ofWisconsin has ordered the SCDP to conduct a series of 
data collection, analysis, and reporting activities focused on evaluating the MPCP as a whole. The 
data collection activities include collecting the test scores from all standardized testing conducted by 
participating MPCP schools and converting those scores into data that can be compared both with 
MPS test scores and over time. The SCDP also is required to draw a representative sample of MPCP 
students and obtain test scores from them on the WKCE in order to make it easier to compare MPCP 
performance with MPS performance. These comparisons and evaluations will be made very carefully, by 
an experienced team of school choice researchers. Under the law, the data and results of the evaluations 
will be forwarded to the Legislative Audit Bureau for secondary analysis and review, and reported to the 
state legislature. 

1.7 What specific testing information do we need to provide to the SCDP? 

The SCDP evaluation team will need the following information about each student that MPCP schools 
test: 



• At least two measures of the student’s scores in the three sections (reading, math, and science). 
Scaled scores are preferred, but Normal Curve Equivalents (NCEs) and National Percentile 
Ranks (NPRs) are also acceptable. For example, a school might provide each student’s scaled 
scores and NCEs in reading, math, and science. Raw scores will not be useful to the evaluators. 

• Basic information about the test that was administered and the student who produced the scores, 
including the name of the test, the date(s) of testing, if any special accommodations were made 
for the student, as well as the name, grade level, birthday and school attended by the student. 

In many cases, all or most of this information will be on the individual student records that the 
testing company provides to schools upon scoring the tests. 

1.8 In what form should we provide that information to the SCDP? 

If your school receives the results of its testing program in electronic format, as a database, then we 
strongly prefer that you forward the information to us electronically. If you receive the results in paper 
form, please make copies of the paper records and send them to us. 

1.9 How should we transmit this information to the SCDP? 
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Because the testing information will include sensitive personal information about your students, you 
will need to take special precautions in providing us with the data. One option is for you to arrange for 
SCDP personnel to pick up the testing information in person. Once you have received and made a copy 
of your testing database or paper records, just call our University of Arkansas office and we will make 
arrangements for a member of our research staff to come by your office to obtain the information. A 
second option is for you to ship the information to the University of Arkansas office of the SCDP using a 
secure method of shipping (e.g. Fedex, UPS, or registered U.S. Mail). 

1.10 Will the names of students and schools be kept confidential? 

Absolutely. This information is being collected in order to evaluate the MPCP as a whole and to learn 
more about the effects of school choice programs like the MPCP. No data will be reported at the 
individual level, and no names of participating individuals or schools will ever be reported to anyone 
outside of the SCDP research team. This research is being overseen by the Institutional Review Board 
(IRB) for Human Subjects Research at the University of Arkansas. The IRB requires that the identity 
of all participants - individuals and schools - be kept strictly confidential in the conduct of academic 
research such as the MPCP longitudinal evaluation. As a result of the study, the public will learn how 
well the choice program is performing, and perhaps how the program might be improved, but no one will 
be given any specific information about the students and schools in the MPCP. The SCDP has strong 
safeguards in place to make sure that complete confidentiality is maintained throughout the study. 

COMPONENT TWO: PROGRAM EVALUATION 

2.1 In addition to the testing for program accountability purposes, what else are private schools 
participating in the MPCP expected to do in support of the study? 

The SCDP research team will need help from private schools in conducting the MPCP impact study 
that is called for in the law, based on the representative sample of MPCP students. Last fall we drew 
a representative sample of approximately 2,500 MPCP students in grades 3-8 and the entire census of 
MPCP 9 th graders. This October we will draw an additional representative sample of 500 3 rd graders. All 
panel students will be administered the same tests taken by their respective public school peers (e.g. the 
WKCE-CRT in grades 3-8 and 10 and the Milwaukee Benchmark Assessment in grade 9). The testing 
of the “MPCP Panel” will take place sometime between October 22 nd and November 23 rd to coincide with 
MPS testing. We will follow this subgroup of MPCP students for another four years, re-testing them 
annually and surveying them and their parents about their educational experiences. The research team will 
handle all of the survey activities, but we will need help from the MPCP schools in testing the MPCP 
panel. We will notify schools early this fall regarding which of their MPCP students have been selected 
for the MPCP panel, and then work with the schools to arrange for the administration of the WKCE- 
CRT and the Benchmark Assessment to panel students during the fall testing window. 

2.2 Who will administer the WKCE-CRT to the students on the MPCP panel and how will they do it? 

We prefer that the WKCE-CRT be administered to students on the MPCP panel in their own school 
by school personnel. Such testing conditions would best replicate how MPS students are tested and 
therefore would facilitate a proper comparison between the performance of MPCP and MPS students. 

The research team will purchase these tests and pay for their scoring. We also will provide training on 
WKCE test administration to the school personnel assigned the responsibility for conducting the testing 
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(training is not necessary for the Milwaukee Benchmark Assessment). Since it is highly unlikely that the 
MPCP panel will include all 3-10 grade students in a particular school, or even all such MPCP students 
in a particular school, we recommend that the testing of the MPCP panel students be conducted on a 
“pull out” basis in order to minimize disruption of the school schedule, and be conducted over several days 
so that students are not spending an entire day away from their classmates and the curriculum. 

Since the entire census of MPCP 9 th graders was selected to participate in the panel last fall, schools 
should expect that close to all of their MPCP 10th graders will be on the panel this year. If a school has a 
few 10 th graders that were not part of the panel last year, but does not wish to exclude those students from 
testing this year, we will provide materials to test all 10 th graders. 

Schools that lack the personnel resources to conduct the testing themselves should contact the Westat 
representative on the SCDP to request assistance. SCDP staff will be available throughout the testing 
period to provide advice, support, and quality control over the test administrations. As soon as the testing 
of the MPCP panelists at a particular school is complete, administrators should contact Westat and a staff 
member will come by to pick up the testing materials for processing. 

2.3 Why are there two testing requirements instead of one? Can the two requirements be satisfied by a 
single testing program? 

There are two testing requirements as a result of Act 125 because one is appropriate for program 
accountability purposes and the other is appropriate for program evaluation purposes. Participating 
schools are required to test all of their MPCP students in grades 4, 8, and 10 using the qualified test they 
choose in order to generate program-wide information about how well MPCP students are doing. That 
information could not serve as the foundation for a reliable study of the effects of the MPCP program on 
student outcomes because the sample is limited to a few grades and the students will be assessed using 
a variety of tests. In order to determine if the MPCP program itself is improving student outcomes, we 
need to follow a representative panel of MPCP students and assess their performance using the same test 
that comparable MPS students take - the WKCE-CRT. In specific schools, the group of students that 
take the tests, and the specific tests that are administered, will likely differ for purposes of accountability 
(all 4 th , 8 th , and 10 th MPCP students on school-selected test) and evaluation (a sample of 3 rd -10 th MPCP 
students on the MPS test). The two testing programs will only perfectly overlap at a particular school if 
both of the following conditions hold: 

• All MPCP students in grades 3-8 (if elementary) or 9 and 10 (if high school) are tested in 
reading, math, and science in the fall of every year; 

• The school administers the public school test (WKCE-CRT if grades 3-8 and 10, Milwaukee 
Benchmark Assessment if grade 9) as its school-selected test. 

Under those conditions, the school could simply provide the SCDP with a copy of the results of its annual 
testing program and the school’s testing requirements under Act 125 would be completely satisfied. Some 
participating MPCP schools might administer the WKCE-CRT as their annual student test, but not 
necessarily to the students in all of the grades 3-8. If that is the case, we will accept the test scores of the 
students that were tested and make arrangements with the school to test any students in the MPCP panel 
attending that school that were not tested. Thus, for private schools that administer the WKCE-CRT, 
there will be overlap between the two testing programs but the overlap might not be complete. 
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2.4 What testing accommodations should we provide for students with special needs? 

We urge test administrators to provide appropriate testing accommodations based on the type and severity 
of special needs that students may have. In the public schools, such accommodations are detailed in a 
students Individualized Education Plan (IEP). Although IEPs are not usually used by private schools, 
if a specific is known to have or to have had an IEP that specified testing accommodations, then those 
accommodations should be followed for purposes of MPCP testing to the extent possible. In most cases, 
MPCP students with disabilities or language challenges that affect their ability to take tests will not have 
IEPs as a guide. In those cases, school personnel must do the following: 

• Notify Westat regarding who will be accommodated, how, and why; 

• Document all accommodations on a sheet sent to the test scoring company along with the testing 
materials (Westat will provide one if the testing company does not); 

• Administer the same set of accommodations to the same students each year that they are tested. 

The state of Wisconsin administers alternate assessments to students with disabilities (the WAA-SwD) 
and for English language learners (WAA-ELL) in the same subjects by grade as the WKCE-CRT. 

Westat will have copies of those alternative assessments available for testing purposes and will train test 
administrators on how to use them. 



CONTACT INFORMATION 

What ifwe have additional questions regarding the testing requirements under Act 125? 

For questions regarding what participating MPCP schools are required to do under the new law, please 
contact the Department of Public Instruction: 

Tricia Collins: 608-266-2853 or toll free 1-888-245-2732, ext 3 then 3 

For questions regarding the specific process of transferring the results of your school’s regular testing 
program to the School Choice Demonstration Project, please contact the School Choice Demonstration 
Project: 

Laura Jensen: 479-575-6345 

For questions regarding the specific process for testing students on the MPCP panel, please contact 
Westat: 

Bonnie Ho: 301-212-2185 
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SCHOOL CHOICE 
DEMONSTRATION PROJECT 

GEORGETOWN UNIVERSITY 



College of Education and Health Professions 
Department of Education Reform 

School Choice Demonstration Project 




UNIVERSITY^ARKANSAS 

February 3rd, 2009 

201 Graduate Education Building 
Fayetteville, Arkansas 72701 
(479) 575-6345 
(479) 575-3196 (FAX) 



Dear Principal: 

As our previous correspondence has mentioned, Wisconsin Act 125 mandates that all schools 
participating in the Milwaukee Parental Choice Program (MPCP) administer annually a nationally 
normed, standardized test of their choosing or the WKCE to their choice students in grades 4, 8, and 10. 
The test must cover reading, math and science. Additionally, Act 125 requires that MPCP schools submit 
a copy of all individual student level scores from any standardized tests they administer to the School 
Choice Demonstration Project (SCDP) at the University of Arkansas. The purpose of this letter is to 
provide you with guidance regarding how to submit your school’s test scores to the SCDP. 

Packaging of Scores 

Once you have received the scores from all your 2008-09 testing, please copy the individual student level 
score sheets and place the copies in the FedEx package you received along with this letter. You will find 
enclosed in this package a copy of the MPCP Principals’ Survey. Please complete this survey and insert it 
in the FedEx package along with your school’s test scores. Next, complete the top portion of the FedEx 
airbill with your school’s name and address. Remove the “sender’s copy” for your records. This copy can 
be used to track the package in the event that it does not arrive at the SCDP. Please either have a FedEx 
representative pick up the package at your school or drop off the package at a FedEx store. 

Time Frame & Mailing Address 

Results of any fall testing that was administered at your school should be mailed to the SCDP by March 
1 st . If your school conducts both fall and spring testing, or only spring testing, these results should be 
mailed to the SCDP by July 1 st . Please wait until you have received the results of ALL 2008-09 testing 
before sending the SCDP your FedEx package. 

Format of Scores 

A copy of the scores can be submitted to the SCDP in either electronic or paper format, though paper 
format is strongly preferred. 

Security Protocols 

To protect the confidentiality of your students, it is important certain data protection strategies be 
implemented. If you are sending an electronic copy of your students’ scores, you must password protect 
the file and burn the file to a CD. Instructions on how to password protect Microsoft Office 97-2003 
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Word, Microsoft Excel, and Acrobat Professional documents can be found below. If you choose to submit 
a paper copy of your students’ test scores, please sign your name over the seal on the FedEx package. 

Securing a Word or Excel file with a password: 

With the file open: 

• On the Tools menu, click Options, and then click Security. 

• In the Password to open box, type MPCP2009, and then click OK. 

• In the Reenter password to open box, type MPCP2009 again, and then click OK. 

Securing an Acrobat Professional/PDF file with a password: 

With the PDF document open: 

• Control D for Document Properties 

• Click Security Tab > Change Security Method to Password Security 

• Check “Require a password to open the document” 

• Enter a password MPCP2009 in the “Document Open Password:” box 

• Confirm the password MPCP2009 

If the electronic file containing your students’ scores is not in any of these formats, please check the help 
menu in the program that fits the file for guidance in how to password protect the file. Another option 
would be to convert the file from its program format into Excel, which is the most preferred format for 
transferring data files securely. 

We look forward to receiving your school’s test scores. As always, if you have any questions, please call 
Laura Jensen or Brian Kisida, SCDP Research Associates, at 479-575-6345. 

Sincerely yours, 



Patrick J. Wolf, Ph.D. 

Professor and Endowed Chair in School Choice 
Principal Investigator, SCDP 
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College of Education and Health Professions 
Department of Education Reform 

School Choice Demonstration Project 



201 Graduate Education Building 
Fayetteville, Arkansas 72701 
(479) 575-6345 
(479) 575-3196 (FAX) 



J uly 15, 2009 



Dear 

I hope this letter finds you doing well and enjoying the summer weather! I want to again thank 
you for all your assistance throughout this year as we gathered data for the third year of the 
state-mandated Longitudinal Evaluation of the Milwaukee Parental Choice Program. 

This letter serves as a final reminder that your school has yet to mail us your principal survey or 
your 2008-09 student test scores. We need a copy of all individual-level test results from ANY 
standardized testing that your school did during this school year. 

On August 2nd, I will notify officials at the Wisconsin Department of Public Instruction to let 
them know which MPCP schools have failed to provide the SCDP with individual level test scores 
for the 2008-09 school year as is mandated in the 2005 Wisconsin Act 125. 

I have included an additional copy of the principal survey for your use. If you have questions 
or need additional FedEx materials to mail in your scores and survey, please contact SCDP 
Research Associate Laura Jensen (479-575-6345). 



Sincerely, 




Patrick Wolf, Ph.D. 

Principal Investigator 

School Choice Demonstration Project 
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A ppendix A: 

Schools with MPCP Students in Tested Grades Operating through May 2009 
N— 1 16; (The three italicized schools did not provide acceptable test scores in 2008-09) 



Alston's Preparatory Academy 


Dr. Brenda Noach Choice School 


LifeSkills Academy 


Atlas Preparatory Academy 


Early View Academy of Excellence 


Lutheran Special School & Education 
Services 


Atonement Lutheran School 


Eastbrook Academy 


Marquette University High School 


Believers in Christ Christian Academy 


Emmaus Lutheran School 


Messmer High School 


Blessed Sacrament School 


Excel Academy 


Messmer Prep Catholic School 


Blessed Savior - East Campus 


Excel Learning Academy 


Mills Christian Academy 


Blessed Savior - North Campus 


Fairview Lutheran School 


Milwaukee Lutheran High School 


Blessed Savior - West Campus 


Family Montessori School 


Milwaukee Montessori School 


Blessed Savior-South Campus 


Garden Homes Community Montessori 






School, Inc. 


Milwaukee Seventh Day Adventist 


Carter's Christian Academy 


Garden Homes Lutheran School 


School 


Catholic East Elementary School 


Gilchrist Christian Academy 


Mother of Good Counsel School 


CEO Leadership Academy 


Gospel Lutheran School 


Mount Calvary Lutheran School 


CeriaM. Travis Academy, Inc. 


Greater Holy Temple Christian Center 


Mount Lebanon Lutheran 


Christ Memorial Lutheran School 


New Testament Christian Academy 


Christ St. Peter Lutheran School 


Harambee Community School 
Hickman Academy Preparatory School 


Northwest Lutheran School 


Christian Faith Academy of Higher 


Notre Dame Middle School 


Learning 


Holy Redeemer Christian Academy 


Oklahoma Avenue Lutheran School 


Clara Mohammed School 


Holy Wisdom Academy 


Our Lady of Good Hope School 


Community Vision Academy LTD 


Hope Christian School 


Our Lady Queen of Peace Parish 


Concordia University School and 


Hope Middle School 


Institute for LIGHT 


Institute ofTechnology and Academics 


Outlook University Independent 


CrossTrainers Academy 


School Network 


Jared C. Bruce Academy 


Parklawn Christian Leadership 


Daughters of the Father Christian 
Academy 


Johnson Christian Academy, Inc. 


Academy 


Destiny High School 


Kidpreneur 


Pius XI High School 


Divine Savior Holy Angels High School 


King's Academy Christian School 


Prince of Peace 




Life 101 "THINK" Institute 


Resurrection Christian Academy 
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Right Step, Inc. 

Risen Savior Lutheran School 
Saint Adalbert School 
Saint Anthony School 
Saint Bernadette School 
Saint Catherine of Alexandria 
Saint Catherine School 
Saint Charles Borromeo School 
Saint Gregory the Great Parish School 
Saint Joan Antida High School 
Saint John Kanty School 
Saint John's Evangelical Lutheran 
Saint Josaphat Parish School 
Saint Leo Catholic Urban Academy 
Saint Lucas Lutheran School 
Saint Marcus Lutheran School 
Saint Margaret Mary School 



Saint Martini Lutheran School 

Saint Peter-lmmanuel Lutheran 
School 

Saint Philip's Lutheran School 

Saint Rafael the Archangel School 

Saint Roman Parish School 

Saint Rose Catholic Urban Academy 

Saint Sebastian School 

Saint Thomas Aquinas Academy 

Saint Vincent Pallotti School 

Salam School 

Sharon Junior Academy 

Sherman Park Lutheran School/ 
Preschool 

Siloah Lutheran School 
Tamarack Community School 
Texas Bufkin Academy 



The Hope School 

The Margaret Howard Christian 
Leadership Institute 

The Way and the Truth Christian 
Academy 

Travis Technology High School 

Trinity Christian Academy for 
Nonviolence 

Tuskegee Aviation Academy 

Urban Day School 

Victory Christian Academy 

Washington DuBois Christian 
Leadership Academy 

Wisconsin Lutheran High School 

Word of Life Evangelical Lutheran 
School 

Yeshiva Elementary School 
Young Minds Preparatory School 
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